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I.  INTRODUCTION 


The  Department  of  Defense  has  become  very  adept  at  managing  assets 
in  both  war  and  peacetime  operations  including  movement  of  logistical  supplies, 
battle  groups,  personnel  and  information  systems.  These  assets  are  tangible 
and  the  metrics  appear  to  be  relatively  well  defined  for  measurement  of 
performance  allowing  adjustments  to  be  made  that  attempt  to  maximize  the 
capabilities  in  any  given  operating  environment.  The  intangible  asset, 
management,  which  is  required  to  ensure  these  important  activities  are  executed 
at  the  appropriate  time  and  place,  is  not  considered  in  the  metrics  that  are 
currently  utilized.  It  follows,  that  questions  can  be  posed,  such  as,  “What  is  the 
value  added  by  information  technology  to  support  Maritime  Domain  Awareness?” 
And,  “How  much  value  does  management  add  to  the  execution  of  these  tasks 
and  what  is  the  Return  on  Investment  (ROI)  for  management’s  involvement?” 

In  order  to  evaluate  the  effectiveness  of  management  and  the 
accompanying  ROI,  a  new  approach  to  measurement  must  be  considered  that 
will  take  into  account  the  value  of  intangible  assets  like  routine,  explicit 
knowledge  and  management’s  unique  creative  intuitive  knowledge  and  convert 
the  results  into  common  units  of  output  across  the  enterprise  to  enable 
comparability  of  estimates.  The  measurement  tool  will  take  into  account  the 
performance  of  organizational  assets  including  personnel,  processes,  and 
technology.  The  approach  must  also  account  for  intangible  assets  such  as 
knowledge,  and  intellectual  capital.  Knowledge  value  added  (KVA)  provides  a 
methodological  framework  for  measuring  the  value  added  of  these  assets  as  they 
convert  inputs  into  outputs.  With  this  approach  it  is  possible  to  describe  all 
process  outputs  in  common  units.  Using  the  traditional  valuation  technique  of 
“market  comparables,”  it  is  possible  to  assign  a  price  per  unit  of  output  that 
enables  the  use  of  common  financial  metrics  such  as  discounted  cash  flow,  net 
present  value,  ROI.  Further,  the  KVA  framework  can  be  used  to  calibrate 

management’s  creative  outputs  in  terms  of  the  amount  of  time  it  takes  an 
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average  manager  to  learn  how  to  make  these  kinds  of  decisions  through  formal 
and  informal  education,  training,  and  experience. 

Using  this  technique,  a  value  can  be  determined  for  intangible  assets  such 
as  management’s  creative  activities  that  have  been  labeled  “management  dark 
matter”  (Housel  and  Kanevsky,  2007).  Further,  this  new  source  of  data  can  be 
used,  not  only  to  conduct  analysis  on  the  value  of  management,  but  may  also  be 
used  to  structure  options  for  future  strategic  planning  within  an  organization. 

The  measurement  framework  selected  was  developed  at  the  Naval 
Postgraduate  School  (NPS)  by  Dr.  Thomas  Housel  and  Dr.  Jonathan  Mun.  The 
approach  utilizes  the  Knowledge  Value  Added  and  Real  Options  (KVA+RO) 
valuation  framework  and  addresses  the  difficulty  in  measuring  intangible  assets 
such  as  management.  The  KVA+RO  analysis  was  originally  designed  to 
measure  IT  portfolio  acquisitions  and  allow  for  the  development  of  real  options 
utilizing  Monte  Carlo  simulation  and  portfolio  optimization  techniques  to  estimate 
the  risk-return  tradeoffs.  This  valuation  allows  decision  makers  to  look  at  the 
options  available  to  them  to  maximize  return  on  investment  while  ensuring 
optimization  of  resources  and  risk  mitigation. 

This  thesis  extends  original  work  by  Housel,  Bell,  and  Kanevsky  on  the 

KVA  framework  by  demonstrating  that  it  can  be  adapted  for  use  in  the  valuation 

of  management  in  the  same  manner  it  is  utilized  to  value  other  productive  assets 

within  processes.  The  research  will  begin  with  a  discussion  of  current 

management  valuation  reports  and  how  they  fail  to  provide  defensible  estimates 

of  the  value  added  by  individual  managers  in  a  comparable,  objective  way  that 

can  be  effectively  utilized  by  decision  makers.  The  KVA+RO  valuation 

framework  will  be  discussed  in  detail  in  the  Second  section  to  demonstrate  how 

this  framework  can  be  used  to  place  a  value  on  all  productive  assets.  The  Third 

section  will  be  a  demonstration  of  the  extension  of  the  KVA  methodology  to 

measure  the  value  added  by  management  using  a  case  study  of  the  Maritime 

Domain  Awareness  (MDA)  Track  Generation  process.  The  case  study  provides 

a  proof  of  concept  of  the  use  of  a  new  measure  of  management’s  value  added 
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within  the  context  of  the  existing  KVA  approach  and  will  demonstrate  the  ability  to 
value  productive  assets,  including  managers,  and  utilize  that  valuation  to 
recognize  the  options  available  to  decision  makers.  This  section  will  also 
demonstrate  how  the  KVA  data  can  be  used  to  inform  the  analysis  of  real 
options.  Section  Four  will  be  a  discussion  of  the  results  from  the  proof  of  concept 
and  how  the  results  can  be  utilized  to  provide  a  metric  for  management  and 
empower  decision  makers  with  options  steering  away  from  subjective  analysis  of 
management  performance  and  replacing  it  with  a  defensible  and  objectively 
derived  quantitative  value.  Section  five  will  discuss  utilizing  the  KVA  approach  to 
measure  the  value  added  by  management  within  the  context  of  the  MDA  proof  of 
concept.  The  discussion  will  cover  how  valuation  of  management  can  be 
approached  using  the  KVA  methodology  and  how  decision  makers  can  be  more 
effective  at  understanding  the  value  of  management  within  an  organization. 
Section  Six  will  discuss  recommendations  for  implementation  of  this  technique, 
limitations  and  future  research  on  the  subject  matter. 
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II.  PRESENT  DAY  IDEAS  OF  VALUATION 


Industry  leaders  recognize  a  need  to  show  the  value  of  management  but 
most  of  the  research  done  focuses  on  whether  or  not  management  has  an  effect 
on  the  overall  organizational  value  or  how  the  value  of  an  organization  can  be 
increased  with  management.  The  research  does  not  look  at  the  effect  of 
individual  managers  on  an  organization  or  objective  metrics  to  measure  the  value 
added  by  those  individual  managers.  The  literature  reviewed  will  be  briefly 
discussed  to  further  demonstrate  the  need  for  a  metric  allowing  us  to  perform  a 
valuation  of  managers  enabling  decision  makers  to  see  how  much  value  a 
manager  adds  to  an  organization. 

“Does  Corporate  Governance  Effect  Firm  Value?”  was  the  question 
posed  by  Black,  Jang,  and  Kim  (Black,  et  al.,  2003).  They  answered  this  question 
by  developing  a  corporate  index  of  526  Korean  companies  and  conducting 
statistical  analysis  to  find  a  correlation  between  corporate  governance  and  firm 
value.  A  key  factor  in  this  research  is  “that  the  index  is  based  on  information 
obtained  on  shareholder  rights,  board  of  directors  in  general,  outside  directors, 
audit  committee  and  internal  auditor  disclosure  to  investors,  and  ownership 
parity”  (Black,  et  al.,  2003,  p.  3).  It  is  important  to  note  the  index  used  in  this 
research  because  it  demonstrates  that  the  manager  as  a  single  entity  is  being 
overlooked  when  calculating  the  effect  management  governance  has  on  an 
organization’s  value.  The  decision  maker  will  only  know  if  the  governance  of  the 
management  team  as  a  whole  had  an  effect  on  a  firm’s  value  and  not  if  the 
management  team  or  a  particular  manager  had  a  significant  contribution  to  the 
organization’s  value.  The  results  of  the  analysis  demonstrated  a  strong  positive 
correlation  between  corporate  governance  and  firm  value  which  emphasizes  that 
decision  makers  need  to  be  able  to  apply  a  metric  to  the  management  team  and 
individual  managers  so  they  will  be  able  to  further  exploit  an  organizations  assets 
and  potentially  further  increase  the  organizational  value. 
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Beiner,  Stefan,  Wolfgang  Drobetz,  Markus  Schmid,  and  Heinz 
Zimmerman  conducted  similar  research  on  corporate  governance  and  the 
valuation  of  specific  firms  which  also  demonstrated  a  positive  relationship.  The 
research  did  not  isolate  any  one  manager  or  management  function.  The 
researchers  used  an  index  constructed  of  Swiss  firms  based  on  a  broad 
corporate  governance  index  and  applied  five  additional  variables  related  to 
ownership  structure,  board  characteristics,  and  leverage  to  provide  a 
comprehensive  description  of  firm-level  corporate  governance  (Beiner,  et  al., 
2006).  The  research  results  show  that  corporate  governance  does  indeed  have 
a  distinguishable  effect  on  firm  value,  but  does  not  allow  a  decision  maker  to 
distinguish  whether  the  management  team  as  a  whole  or  individual  manager 
contributed  to  the  additional  firm  value.  The  decision  maker  is  forced  to  assume 
that  the  team  as  a  whole  contributed  to  the  increased  firm  value  and  in  actuality  it 
may  only  be  a  small  portion  of  the  management  team’s  contributions  that  are 
causing  the  value  to  increase. 

A  slightly  different  approach  was  used  in  Klein,  Peter,  Shapiro,  and 
Young’s  attempt  to  assess  a  correlation  between  corporate  governance  and  firm 
value  with  an  emphasis  on  family  ownership  of  a  company.  The  companies  used 
in  the  research  were  Canadian,  but  in  many  cases  were  traded  on  a  United 
States  Stock  Exchange  as  well.  When  looking  for  variables  to  conduct  the 
analysis  the  authors  noted  the  difficulty  in  finding  good  “instruments”  to  work  with 
for  assessing  the  correlation  of  firm  value  based  on  corporate  governance  (Klein, 
et  al.,  2005).  The  two  variables  that  were  selected  were  whether  or  not  the 
company  was  traded  in  a  United  States  exchange  and  how  long  the  company 
had  been  traded  on  the  Toronto  Stock  Exchange.  The  results  of  the  research  did 
not  demonstrate  a  positive  correlation  as  with  the  previous  two  studies  and  in  fact 
the  researchers  found  that  corporate  governance  did  not  seem  to  be  a  factor  for 
the  companies  in  Canada.  The  data  would  be  difficult  information  for  a  decision 
maker  within  a  corporation  to  comprehend  because  it  would  not  allow  for  a 
reasonable  assessment  of  the  effectiveness  of  the  firm’s  management  team  and 
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how  to  address  either  increasing  or  decreasing  organizational  value.  The 
question  might  arise:  Is  a  management  team  required  for  the  organization  if  this 
study  was  indeed  accurate?  The  only  way  to  truly  know  the  effectiveness  of  a 
management  team  or  manager  is  to  independently  find  a  way  to  measure  that 
intangible  entity  separating  it  from  other  factors  such  as  corporate  governance. 

Hsiu  Ting  conducted  research  on  corporate  governance  in  Taiwan  utilizing 
the  factor  of  poor  economic  conditions.  The  first  thing  Ting  did  was  acknowledge 
that  some  previous  studies  found  that  corporate  governance  did  not  have  a  factor 
on  firm  value  and  then  conducted  his  own  analysis  to  confirm  whether  or  not 
there  was  a  correlation.  Ting  conducted  statistical  analysis  of  207  companies 
utilizing  a  simple  summary  factor  and  concluded  that  there  was  indeed  a 
correlation  between  corporate  governance  and  firm  value.  The  research  further 
found  that  “firms  with  poor  corporate  governance  mechanism  tend  to  perform 
badly  when  business  cycle  goes  downward”  (Ting,  2006).  The  most  important 
find  related  to  this  thesis  research  is  that  Ting  also  found  that  “the  recognition  of 
supervisor  is  an  important  factor  for  corporate  governance  effect  as  well”  (Ting, 
2006).  This  factor  is  important  because  it  emphasizes  the  need  for  a  metric  that 
can  be  used  to  assess  a  manager’s  performance  and  the  assumption  can  be 
made  that  the  performance  does  contribute  directly  to  the  value  of  a  firm. 

Michael  Armstrong  has  written  a  guide  for  managers  to  utilize  called  A 
Handbook  of  Management  Techniques.  The  guide  specifically  talks  about  value 
adding  skills  and  technique  for  managers  and  breaks  them  up  into  different 
management  specializations.  The  driving  force  through  the  guide  is  to  improve 
the  manager  through  better  decision  making,  various  methodologies  leading  to 
improved  effectiveness.  The  dilemma  is  how  does  a  decision  maker  assess  how 
much  added  value  to  the  organization  will  the  manager  be  or  has  become  without 
the  metric  to  measure  his  or  her  performance.1 

1  Thomas  Housel  and  Kanevsky,  Valery,  “Measuring  the  Value  Added  of  Management:  A 
Knowledge  Value  Added  Approach”,  Working  Paper,  Not  Yet  Published,  2007 
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This  thesis  encompasses  key  components  in  the  development  of  a 
methodology  to  measure  Management  Value  Added  (MVA)  and  begins  with  the 
Knowledge  Value  Added  (KVA)  methodology  described  within  the  book 
Measuring  and  Managing  Knowledge  by  Dr.  Thomas  Housel  and  Dr.  Arthur  Bell 
(2001 ).  The  literature  discusses  the  shift  in  paradigm  as  we  enter  the  information 
age  and  the  importance  of  knowledge  being  managed  as  an  asset  even  though  it 
is  considered  an  intangible  asset.  An  important  concept  that  is  stressed  in  this 
literature  is  the  assumption  that  there  is  a  direct  relationship  between  knowledge 
and  the  value  it  creates  (Housel  and  Bell,  2001).  This  concept  allows  the  current 
thesis  research  to  demonstrate  the  idea  that  knowledge  does  equate  to  value 
and  that  if  management  implicit  knowledge  can  be  described  then  it  can  be 
measured  and  translated  into  a  value  that  can  be  associated  with  the  cumulative 
value  produced  by  an  organization.  The  Knowledge  Value  Added  (KVA) 
approach  was  used  to  conduct  a  case  study  as  part  of  the  research  for  this 
thesis.  This  approach  is  explained  in  the  literature  as  a  methodology  to  estimate 
the  value  of  knowledge  deployed  throughout  a  company’s  core  processes 
(Housel  and  Bell,  2001).  The  value  added  is  calculated  by  utilizing  a  “return  ratio 
with  the  numerator  of  the  ratio  being  the  percentage  of  the  revenue  or  sales 
dollar  allocated  to  the  amount  of  knowledge  required  to  obtain  the  outputs  of  a 
given  process  in  proportion  to  the  total  amount  of  knowledge  required  to 
generate  the  corporation’s  salable  outputs”  (Housel  and  Bell,  2001,  p.  40).  The 
denominator  of  the  ratio  is  the  cost  to  execute  the  process  knowledge. 

The  next  piece  of  key  literature  supporting  this  research  is  Real  Options 
Analysis  by  Dr.  Johnathan  Mun.  Real  Options  theory  allows  an  organization  to 
make  decisions  looking  at  all  options  available  to  them.  “Traditional  discounted 
cash  flow  approaches  assumes  a  single  decision  pathway  with  fixed  outcomes, 
and  all  decisions  are  made  in  the  beginning  without  the  ability  to  change  and 
develop  over  time”  (Mun,  2006,  p.  92).  The  theory  utilizes  the  value  we  are  able 
to  apply  to  knowledge  through  Knowledge  Value  Added  methodology  and 
conduct  Monte  Carlo  simulation  which  ultimately  provides  decision  makers  with 
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multiple  options  for  an  organization.  The  options  may  be  simplistic  in  nature  or 
give  the  organization  numerous  options  allowing  the  organization  to  execute 
projects  at  the  most  advantageous  time  to  gain  maximum  benefit  and  value. 
Real  Options  theory  is  the  second  component  to  the  KVA+RO  framework  utilized 
in  the  case  study  to  demonstrate  the  ability  for  one  to  place  value  on  an 
intangible  asset  such  as  knowledge  and  utilizes  this  asset  in  decision  making  to 
increase  the  value  of  an  organization. 

The  next  step  is  to  lay  the  groundwork  for  utilizing  a  similar  approach  to 
value  management  implicit  knowledge  as  an  intangible  asset  and  develop  a 
metric  to  consistently  measure  that  value  against  the  organizational  value  to 
determine  how  much  contribution  can  be  associated  with  the  manager  and/or 
management.  The  cornerstone  piece  of  literature  for  this  thesis  is  a  white  paper 
titled  Measuring  the  Value  Added  of  Management:  A  Knowledge  Value  Added 
Approach  by  Dr.  Thomas  Housel  and  Dr.  Valery  Kanevsky  (2007).  The  working 
paper  looks  at  the  increasing  use  of  an  open  business  model  used  in 
organizations  today.  The  model  attempts  to  address  the  increasing  burden  on 
management  based  on  the  amount  of  complexity  managers  must  deal  with  by 
attempting  to  mitigate  risks,  resolve  uncertainties,  improve  predictions,  and 
exercise  the  control  and  oversight  necessary  to  be  successful  (Housel  and 
Kanevsky,  2007). 

In  relation  to  this  thesis  work,  this  aspect  is  important  as  we  address  the 
value  of  managers  and  how  decision  makers  can  become  empowered  with  a 
metric  to  see  how  well  a  manager  is  performing  in  the  increasingly  complex 
environment  and  be  able  to  correlate  that  to  the  overall  value  of  the  organization. 
“The  idea  of  management  dark  matter  is  introduced  in  this  literature  as  the  use  of 
manager’s  creative  insights  when  they  attempt  to  predict  the  future,  create 
potential  pathways  to  accomplish  the  predictions,  and  control  for  future  risks. 
Those  activities  that  are  uniguely  associated  with  management  involve  the 
creative  use  of  decision  heuristics  based  on  their  implicit  knowledge  accumulated 
over  years  of  experience,  training  and  education”  (Housel  and  Kanevsky,  2007, 
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p.  5).  This  concept  is  important  to  understand  as  we  help  to  further  this 
methodology  with  the  case  study  and  explanation  of  how  the  Management  Value 
Added  approach  can  be  applied. 
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III.  OVERVIEW:  KVA+RO  FRAMEWORK 


“KVA  measures  the  value  provided  by  human  capital  assets  and  IT  assets 
by  analyzing  an  organization,  process  or  function  at  the  process-level”  (Housel 
and  Mun,  2007,  p.  7).  It  provides  insights  into  each  dollar  of  IT  investment  by 
monetizing  the  outputs  of  all  assets,  including  intangible  knowledge  assets.  An 
output  is  defined  as  the  end  result  of  an  organization’s  operations;  it  can  be  a 
product  or  service  as  shown  in  Figure  1 . 


Figure  1.  Measuring  Output  (From:  Uchytil,  2006) 


KVA  is  designed  to  assist  organizations  such  as  the  Department  of 
Defense  manage  IT  investments  and  risk  mitigation.  The  framework’s  key 
components  are  data  collection,  KVA  methodology,  and  Real  Options  analysis 
that  collectively  provide  the  basis  for  performance  data  and  analysis  on  individual 
projects,  programs  and  processes  within  a  portfolio  of  IT  investments  (Housel 
and  Bell,  2001).  There  are  three  methods  that  KVA  analysis  can  be  conducted, 
as  shown  in  the  Table  1 . 
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Table  1.  Three  Approaches  to  KVA  (From:  Housel  and  Bell,  2001) 


Steps 

Learning  Time 

Process  Description 

Binary  Query  Method 

1 

Identify  core  process  and  its  sub  processes. 

2 

Establish  common  units 
to  measure  learning 
time 

Describe  products  in  terms  of  instructions  required  to 
reproduce  them  and  select  unit  of  process 
description. 

Create  set  of  binary  yes/no  questions  such 
that  all  possible  outputs  are  represented  as 
sequence  of  yes/no  answers. 

3 

Calculate  learning  time 
to  execute  each  sub 
process. 

Calculate  number  of  process  instructions  pertaining  to 
each  sub  process. 

Calculate  length  of  sequence  of  yes/no 
answers  for  each  sub  process. 

4 

Designate  sampling  period  long  enough  to  capture 
representative  sample  of  core  process's  final 
product/service  output. 

5 

Multiply  learning  time  for 
each  sub  process  by 
number  of  times  sub 
process  executes 
during  sample  period. 

Multiply  number  of  process  instructions  used  to 
describe  each  sub  process  by  number  of  times  sub 
process  executes  during  sample  period. 

Multiply  length  of  yes/no  string  for  each  sub 
process  by  number  of  times  this  sub 
process  executes  during  sample  period. 

6 

Allocate  revenue  to  sub  processes  in  proportion  to 
quantities  generated  by  Step  5  and  calculate  costs  for 
each  sub  process. 

7 

Calculate  ROK  and  interpret  results. 

A.  DATA  COLLECTION:  KVA  METHODOLOGY 


The  KVA  methodology  was  used  to  fulfill  the  requirements  for  this  thesis 
as  the  research  project  progressed  and  Maritime  Domain  Awareness  (MDA) 
Track  Generation  processes  unfolded  over  a  period  of  approximately  eleven 
months.  The  initial  phase  began  with  Knowledge  Value  Added  (KVA)  data 
collection  obtained  through  interviews  with  MDA  management  and  MDA  process 
Subject  Matter  Experts  (SMEs).  Observations  were  made  to  obtain  average 
learning-time  estimates  and  the  number  of  roughly  equivalent  MDA  process 
instructions  required  to  complete  each  sub-process.  There  was  an  evaluation  of 
the  framework  for  measuring  the  value  of  cost  and  benefit  of  each  system  within 
each  Maritime  Domain  Awareness  (MDA)  process  and  sub-processes  required  to 
produce  an  output.  Once  MDA  process  data  was  accurately  documented,  it  was 
supplemented  by  additional  research  to  compare  cost  of  each  MDA  process  and 
revenue  data  to  establish  baseline  information.  KVA  methodology  was  then 
applied  to  uncover  the  value  of  Maritime  Domain  Awareness  (MDA)  Track 
Generation  and  historical  costs  for  each  process.  Cost  per  unit  of  output  was 
calculated  by  KVA  in  conjunction  with  price  per  unit  estimates  (based  on  a 
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market  comparables  analysis)  that  provided  raw  data  required  for  ROI  analysis 
(Housel  and  Bell,  2001). 

B.  GAUSSSOFT  KVA  SOFTWARE 

The  second  phase  involved  the  use  of  GaussSoft  KVA  software  that  was 
utilized  to  extract  an  array  of  KVA  data  input:  Cost  of  Employees,  Revenues, 
Total  Learning  Time,  Information  Technology,  and  Times  Fired.  A  predetermined 
set  of  Outputs  (Track  Generation,  Track  Quality  Assurance,  Data  Cleansing,  and 
Data  Acquisition),  were  used  to  make  strategic,  and  operational  decisions  in 
analyzing  the  economic  performance  and  cost-benefit  relation  associated  with 
the  Maritime  Domain  Track  Generation  operational  processes. 

In  Figure  2,  there  is  a  profitability  model  created  with  GaussSoft  KVA 
software  for  the  Maritime  Domain  Awareness  (MDA)  process.  The  resources 
contained  the  knowledge  to  produce  outputs. 


Figure  2.  Profitability  Model 


Time  to  Learn  describes  the  amount  of  time  it  takes  an  average  person  to 
learn  a  process.  The  Time  to  Learn  Model  indicates  13.33  weeks  for  Track 
Generation,  3.33  weeks  for  Track  Quality  Assurance,  2.22  weeks  for  Data 
Cleansing,  and  1.11  weeks  for  Data  Acquisition,  as  shown  in  Figure  3.  So  an 
average  MDA  watch  stander  would  take  longer  to  learn  the  Track  Generation 
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portion  of  the  MDA  process  in  order  to  execute  it  properly.  It  takes  less  time  to 
learn  Data  Acquisition. 


Time  to  Learn 


Figure  3.  AS-IS  Time  to  Learn  Model 


The  KVA  model  indicates  a  Track  Generation  of  4.47m,  Data  Cleansing  of 
828.8k,  Track  Quality  Assurance  of  372.96k,  and  Data  Acquisition  of  4.24k,  as 
shown  in  Figure  4.  Even  though  Track  Generation  takes  a  longer  time  to  learn,  it 
returns  a  higher  value  of  knowledge  using  KVA  methodology,  but  a  lower  return 
on  Data  Acquisition. 


KVA 


Figure  4.  KVA  Model 
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Market  Comparables  is  the  valuation  approach  that  can  be  used  to 
estimate  price  per  common  unit,  based  on  market  comparable  revenue  for 
process  outputs.  The  Market  Comparable  model  indicates  a  Track  Generation  of 
$69. 7k,  Data  Cleansing  of  $5. 85k,  Track  Quality  Assurance  of  $-2. Ik,  and  Data 
Acquisition  of  $-8. 5k,  as  shown  in  Figure  5.  Track  Generation  and  Data 
Cleansing  shows  the  most  value  in  the  Maritime  Domain  Awareness 
organization.  Track  Quality  Assurance  and  Data  Acquisition  take  a  substantial 
loss  of  MDA  process  outputs. 


Market  Comp 


MARKET  COMP 


Figure  5.  Market  Comparable  Model 


C.  REAL  OPTIONS 

Real  Options  analysis  incorporates  strategic  planning,  risk  assessment 
and  management,  and  investment  analysis.  Strategic  planning  affords  decision¬ 
makers  the  ability  to  leverage  uncertainty,  limit  risk,  and  generate  numerous 
options  to  increase  the  value  of  a  project  while  managing  the  risk  mitigation 
(Mun,  2006).  As  a  financial  valuation  tool,  Real  Options  allows  organizations  like 
the  Department  of  Defense  to  adapt  to  decisions  of  response  to  unexpected 
environmental  or  market  developments  (Mun,  2006). 

Figure  5  shows  Dr.  Johnathan  Mun’s  integrated  risk  analysis  process  up 
close  into  eight  simple  steps: 
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1 .  Qualitative  management  screening. 

2.  Time-series  and  regression  forecasting. 

3.  Base  case  KVA  and  net  present  value  analysis. 

4.  Monte  Carlo  simulation. 

5.  Real  options  problem  framing. 

6.  Real  options  modeling  and  analysis. 

7.  Portfolio  and  resource  optimization. 

8.  Reporting  and  update  analysis 


< 

o 


W 

a 


1 .  List  of  the 
strategies  to 
evaluate 


«  Sta  rt  with  a  1 1st  of  p  otenti  a  I 
05  p  ro je  cts  o  r  strateg  ies  to  b  e 
reviewed  and  courses  of 
actions  (COA)  to  be 
evaluated... 


2.  Base  case  projections  for 
^  each  project 

2  Data  Gathering 


. . .  project  prospective 
conditions  with  the 
assistance  of  time-series 
forecasting,  comparable 
data,  historical  data,  and 
simulated  data... 


3.  Develop  static  financial 


models  with  KVA  data 

o  _ 


fed 


oi 


.  generate  a  series  of  base  case 
KVA  financial  models  for  each 
strategic  COA  based  on  the 
workflow  processes  involved... 


4.  Risk-based  Monte 
Carlo  simulation 


S-  5?  Simulation 

-§  >- 


...apply  sensitivity  and  scenario 
analysis  coupled  with  Monte  Carlo 
simulation  to  the  analysis  to  quantify 
the  risk  and  uncertainties  involved  in 
each  element  of  the  strategies... 


5.  Framing  Real  Options  6'  °Ptlons  analysis  and  COA 
quantification 

O  Simulation  Lattice 

1  'j*  <# 

£ 

^  Closed-Form  Models 

2  — = /i(.s  t)+az  4st 

. .  .the  relevant  COA  strategies  real  °Ptions  analytics  are  calculated 
are  framed  in  a  strategic  real  through  binomial  lattices  and  closed- 
o  pti  ons  a  na  lys  is.  fo  rm  pa  rti  a  l-d  iffe  renti  al  mo  de  Is  with 


..  .stochastic  optimization  is  the  next 
(optional)  step  if  multiple  projects  exist  that 
requires  efficient  asset  allocation  given 
some  budgetary  constraints...  useful  for 
strategic  portfolio  management... 


8.  Reports  presentation 
and  update  analysis 


create  reports,  make 
decisions,  and  do  it  all  again 
iteratively  overtime... 


Figure  6.  Integrated  Risk  Analysis  Process  (From:  Housel  and  Mun,  2007) 


In  the  ensuing  stages  of  the  framework,  Real  Options  in  the  case  of  this 
thesis  examined  the  Maritime  Domain  Awareness  (MDA)  potential  opportunity  for 
an  investment  which  identified  the  value  of  the  MDA  process  options  through 
ongoing  investments  incorporating  the  outcomes  and  flexibility  in  decisions.  Dr. 
Johnathan  Mun’s  Real  Options  software  was  utilized  using  the  Lattice  and  Monte 
Carlo  simulation  portion.  The  MDA  process  was  broken  down  into  four  options: 

Strategy  A:  AS-IS  option  was  used  to  evaluate  the  current  system  in 
place. 

Strategy  B:  Implementing  the  TO-BE  all  at  once  for  immediate 
development. 
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Strategy  C:  Implementing  the  MDA  process  in  two  different  stages  from 
both  the  East  and  West  Coast. 

Strategy  D:  Implementing  the  TO-BE  Radical  Development. 

There  was  a  set  of  inputs  utilized  for  the  above  options:  PV  asset,  Cost  to 
Execute,  Timing  and  Volatility.  We  used  a  discount  rate  of  5%  and  present  value 
of  asset  of  20  years  of  revenue.  Cost  to  Execute  was  based  on  a  scalar  factor  of 
how  much  the  revenue  would  be.  Timing  was  based  on  the  assumption  that  the 
system  will  last  20  years.  Volatility  was  based  on  simulation  of  various  average 
learning  times  and  the  standard  deviation  divided  by  the  mean  value.  By 
applying  the  Monte  Carlo  simulation  to  these  options,  it  allowed  change  to  all 
major  inputs  in  an  efficient  manner  within  this  model,  and  the  ability  to  identify, 
quantify,  and  analyze  risk  mitigation.  The  resulting  values  were  compared  to 
various  MDA  scenarios  to  assess  potential  improvements  and  how  it  could  be 
applied  in  a  broader  sense  to  management,  and  whether  or  not  it  brings  added 
value  to  any  given  Maritime  Domain  Awareness  process.  In  Chapter  IV,  the 
results  will  show  the  proof  of  concept  and  the  value  of  KVA+RO  analysis. 
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IV.  KVA+RO  RESULTS 


The  KVA  analysis  of  the  MDA  implementation  revealed  that  the  radical 
approach  involving  the  implementation  of  the  developed  software  and  associated 
systems  on  both  East  and  West  Coast  Maritime  Intelligence  Fusion  Centers  will 
vastly  improve  the  Return  on  Investment  (ROI)  for  the  implementation  and 
enable  the  fusion  centers  to  increase  the  volume  of  data  they  are  currently  able 
to  handle. 

A.  SUBSTANTIAL  REVENUE  GENERATION 


The  investment  in  the  development  of  this  is  minimal  in  comparison  to  the 
potential  generated  revenues  from  the  implementation  of  the  track  generation 
software  developed  by  Naval  Research  Laboratories.  Table  2  below  provides 
estimated  revenues  based  on  the  data  collection  from  the  track  generation 
system  in  live  operation  during  both  the  “AS  IS”  and  single  phase  TO-BE 
implementation.  The  potential  for  the  second  phase  (Pacific)  implementation 
increasing  the  volume  handling  capacity  of  the  system  allows  for  an  exponential 
increase  in  track  generation  which  correlates  to  a  substantial  revenue  increase. 


Table  2.  KVA  Results  -  Analysis  of  Revenues  (Annualized) 


Process  Title 

"AS  IS" 

"TO  BE" 

"RADICALTO  BE" 

“AS  IS”  & 
“TO  BE” 
Revenue 
Differential 

“AS  IS”  & 
“RADICAL" 
Revenue 
Differential 

1 

Data  Acquisition 

$2,713.94 

$10,847,273.98 

$397,161,466.92 

$10,844,560.04 

$397,158,752.98 

2 

Data  Cleansing 

$529,548.62 

$2,277,927.54 

$83,403,908.05 

$1,748,378.92 

$82,874,359.43 

3 

Track  Generation 

$953,187.52 

$569,481.88 

$20,850,977.01 

($383,705.64) 

$19,897,789.49 

4 

Track  QA 

$238,296.88 

$1,016,931.94 

$37,233,887.52 

$778,635.06 

$36,995,590.64 

Totals 

$1,723,746.96 

$14,711,615.34 

$538,650,239.51 

$12,987,868.38 

$536,926,492.55 

B.  IMPROVED  PROCESS  PERFORMANCE 


All  of  the  sub-processes  are  improved  with  the  implementation  of  the 
developed  information  technology  but  the  most  important  sub-process  to  note  is 
track  generation.  Figure  7  shows  that  track  generation  is  significantly  increased 
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with  the  implementation  of  the  new  track  generation  technology  with  track 
generation  increasing  by  over  two  hundred  times  the  capability  of  the  AS-IS 
system.  The  significance  of  the  increased  number  of  tracks  is  that  the  amount  of 
information  available  to  the  intelligence  community  customer  is  vastly  increased 
allowing  for  improved  situational  awareness. 


■i  n  n  p  n  o 

12  3 

AS-IS  TO-BE  RADICAL 

Figure  7.  Tracks  Generated  (per  week) 

C.  RETURN  ON  KNOWLEDGE 

The  Return  on  Knowledge  (ROK)  metric  allows  for  value  to  be  placed  on 
the  knowledge  contained  within  both  the  information  systems  and  the  personnel 
who  are  a  part  of  the  process  and/or  sub-processes.  The  metric  allows  for  the 
conversion  of  that  knowledge  into  a  quantifiable  unit  which  empowers  decision 
makers  to  make  informed  decisions  on  processes  based  on  the  value  which  it 
holds.  The  KVA  analysis  demonstrates  the  ROK  for  the  implementation  of  the 
newly  developed  technology  significantly  increases  with  both  the  TO-BE  and 
Radical  implementation.  Table  3  provides  a  break  down  of  the  ROK  for  each 
implementation  broken  down  by  each  sub-process. 
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Table  3.  KVA  Results  -  Analysis  on  ROK 


Core 

Process 

Process  Title 

"AS  IS"  ROK 

"TO  BE"  ROK 

"RADICAL"  ROK 

1 

Data  Acquisition 

1.21% 

4824.52% 

176544.52% 

2 

Data  Cleansing 

235.53% 

1013.15% 

36995.35% 

3 

Track  Generation 

423.95% 

253.29% 

9173.84% 

4 

T rack  QA 

105.99% 

452.30% 

16460.42% 

The  increase  in  ROK  indicates  more  efficient  firing  of  the  knowledge  within 
each  process.  The  large  percentages  may  seem  exaggerated  but  they  are  a 
reflection  of  improved  knowledge  utilization  contributed  to  the  new  technology 
implementation. 

The  application  of  KVA  allowed  discounted  cash  flow  estimates  that  make 
Real  Options  analysis  feasible.  The  Real  Options  framework  uses  a  statistical 
analysis  risk  management  approach  to  estimating  the  risk-rewards  of  the  four 
options  that  were  being  considered  for  this  project.  The  data  was  placed  in  a 
lattice  structured  after  the  options  being  considered  were  selected.  These  options 
included:  leaving  the  system  in  an  AS-IS  state,  immediate  development  of  new 
technology,  two  stage  development  of  new  technology  and  radical  development 
and  deployment  of  the  technology.  The  statistical  analysis  conducted  also 
known  as  Monte  Carlo  simulation  included  five  thousand  trials  to  ensure  reliable 
results.  The  key  for  real  options  analysis  is  to  find  the  value  for  each  option  and 
enable  the  decision  maker  to  make  informed  decisions  that  increase  the  potential 
Return  on  Investment. 

The  results  indicated  that  strategy  D,  radical  development,  was  the  best 
strategy  to  follow  of  the  four  options  considered.  The  results  account  for 
increased  volume  of  contacts  based  on  overall  improved  system  performance 
from  the  newly  implemented  technology.  The  results  from  the  Real  Options 
analysis  are  depicted  in  Table  4. 
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Table  4.  Real  Options  Analysis 


Options 

NPV 

Total  Strategic  Value 

Strategy  A 

6.3  Million 

6.3  Million 

Strategy  B 

121.9  Million 

140.5  Million 

Strategy  C 

51.6  Million 

60.1  Million 

Strategy  D 

14.9  Billion 

25.0  Billion 
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V.  MEASURING  MANAGEMENT 


The  KVA  analysis  of  the  MDA  track  generation  process  options  provide 
proof  of  concept  that  intangible  assets  can  be  accounted  for  within  an 
organization.  The  question  remains  how  we  account  for  the  dark  matter  or 
intuitive  (i.e.,  non-algorithmically  definable)  heuristics  that  allow  a  manager  to 
make  creative  management  decisions  that  predict  the  future  and  potentially  add 
value  to  an  organization?  Dark  matter  accounts  for  a  portion  of  the  manager’s 
activity  and  includes  “the  use  of  managers’  creative  insights  when  they  attempt  to 
predict  the  future,  create  potential  pathways  to  accomplish  the  predictions,  and 
control  for  future  risks  and  uncertainties.  Those  activities  that  are  uniquely 
associated  with  management  involve  the  creative  use  of  decision  heuristics 
based  on  their  implicit  knowledge  accumulated  over  years  of  experience,  training 
and  education”  (Housel  and  Kanevsky,  2007,  p.  3).  Figure  8  shows  a  track 
generation  process. 
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Figure  8.  Track  Generation 

Management  Value  Added  is  structured  within  the  methodology  of  KVA 
and  accounts  for  the  dark  matter  outputs  from  manager’s  creative  knowledge  and 
allows  us  to  estimate  the  amount  of  dark  matter  output  for  each  manager. 
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The  example  that  will  be  used  to  demonstrate  this  concept  is  the 
management  of  the  track  generation  within  the  Maritime  Fusion  Intel  Centers. 
The  watch  officer  and  watch  supervisor  will  generate  dark  matter  outputs 
continuously  while  managing  tracks  generation  to: 

•  mitigate  the  risk  of  false  information, 

•  avoid  poor  fusion  of  data  points  into  tracks, 

•  identify  tracks  that  require  immediate  attention 

•  notify  those  that  have  a  need  to  know  and  understand  the  legalities 
of  the  information  being  generated. 

To  be  successful  managers  of  the  track  generation  process,  they  must 
utilize  their  dark  matter  along  with  their  routine  management  activities  to  ensure 
timely  delivery  of  a  quality  intelligence  product  to  customers  in  an  operational 
environment. 

We  measured  the  dark  matter  outputs  that  the  watch  officer  utilized  by: 

1 .  First,  segmenting  their  job  description  into  “dark  matter  required” 
activities  (e.g.,  finding  a  discrepancy  or  ambiguity  in  generated 
tracks  based  on  their  experience  when  they  “smell”  something 
wrong)  and  routine  management  activities  (e.g.,  QA  check  track 
reports,  posting  track  information  to  various  web  sites) 

2.  Second,  estimate  the  learning  time  required  to  teach  a  novice 
manager  how  to  perform  each  routine  and  dark  matter  based 
activity 

3.  Third,  the  number  of  times  that  the  dark  matter  outputs  were 
generated  within  100,800  tracks  generated  (see  Tracks  Generated 
figure  xx)  per  week 

The  results  indicated  that  of  the  100800  tracks  generated,  6%  raised 
concerns  based  on  the  managers  experience  that  required  further  investigation. 
They  indicated  that  these  tracks  just  didn’t  “smell  right”  (i.e. ,  made  non- 
navigational  sense).  Identifying  these  problematic  tracks  required  managers  to 
use  their  intuition,  implicit  knowledge  to  predict  a  possible  false  identification  of  a 
track.  This  dark  matter  ability  to  predict  a  possible  problem  with  a  track  is  critical 
because  it  dramatically  affects  situation  awareness  of  the  customers  of  this 
information  in  the  operational  environment.  For  example,  an  experience  track 
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manager  would  recognize  that  certain  tracks  were  false  tracks  and  not  a  contact 
of  interest  thus  improving  the  allocation  of  scarce  operational  units  to  investigate 
and  monitor  actual  contacts  of  interest. 

The  amount  of  learning  time  required  to  teach  new  managers  with  college 
degrees  how  to  “smell”  problematic  tracks  was  six  months  of  on  the  job  training 
and  one  week  of  formal  training  or  25  weeks.  This  dark  matter  knowledge  fired 
6000  times  per  week  multiplied  by  25  units  of  learning  time  equals  150,000  units 
of  dark  matter  output.  By  multiplying  the  price  per  unit  of  outputs,  i.e. ,  $.0136, 
times  the  total  dark  matter  outputs  of  150,000  the  value  of  the  dark  matter  activity 
of  the  managers  was  estimated  to  be  $2040  per  week.  This  represents  the 
estimated  value  of  the  track  manager’s  dark  matter.  Table  5  shows  the  results  of 
the  MVA  test  concept. 


Table  5.  MVA  Results  -  Simple  Concept  Test 


Learning  Time 
(in  weeks) 

Dark  Matter  Fired 

Total  Units 

Cost  Per  Unit 

Estimated  Value 
(per  week) 

25 

6000 

150,000 

$.0136 

$2040 

This  is  a  test  of  concept  of  the  approach  to  estimating  the  value  added  by 
manager’s  dark  matter  in  the  context  of  this  case  example.  The  example 
provides  a  metric  in  which  decision  makers  can  assess  the  value  added  by 
managers  to  an  organization  and  estimate  the  potential  and  historical  revenues 
that  might  be  generated  by  the  firing  of  the  manager’s  dark  matter.  This 
information  allows  decision  makers  to  look  at  future  strategies  and  improve  the 
utilization  of  management  to  continue  to  increase  the  value  added  to  the 
organization.  The  example  is  simple  demonstration  of  the  application  in  this 
breakthrough  concept  and  requires  further  testing  to  include  the  use  of  the 
correlation  method  as  discussed  in  Dr.  Thomas  Housel  and  Dr.  Valery 
Kanevsky’s  white  paper.  The  results  could  potentially  be  more  dramatic  in  a 
more  turbulent  environment,  e.g.  point  of  the  spear  battle  space. 
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This  research  calls  into  the  question  the  accuracy  of  KVA  based  estimates 
that  did  not  include  dark  matter.  However,  the  possibility  exists  that  all  the 
outputs  were  captured  in  the  KVA  based  estimates  because  the  dark  matter 
outputs  are  realized  in  the  final  product. 
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VI.  LIMITATIONS/FUTURE  RESEARCH  AND 
RECOMMENDATIONS 


A.  LIMITATIONS 

There  should  be  more  data  collected  over  time  for  the  proof  of  concept 
methodology  and  framework  providing  the  value  of  management  in  the  MDA 
organization,  so  we  could  move  towards  correlation  of  further  testing  for 
validation  of  the  MDA  management,  as  well  as  their  supportive  operational 
leaders. 

B.  FUTURE  RESEARCH 

Future  Research  should  include  overcoming  the  limitations  by  doing  more 
research  in  more  turbulent  domains  and  pursue  correlation  test  approach  that 
was  not  utilized  in  this  thesis  (see  Housel  and  Kanevsky,  2007).  There  are  future 
projects  in  the  works  that  include: 

1.  KVA+RO  framework  solution  to  monitor  processes  for  PEOIWS-1 
beginning  with  data  collection  of  core  processes  implementing 
GaussSoft  KVA  software.  The  data  collection  will  allow  the 
generation  of  performance  parameters  that  will  be  used  for  Real 
Options  analysis. 

2.  Future  Research  utilizing  KVA+RO  framework  to  assess  the  value 
of  knowledge  learned  from  continuous  research  utilizing 
FORCENET,  which  is  the  future  Command  and  Control  system  that 
will  provide  more  robust,  reliable,  and  accurate  Common 
Operational  Picture  (COP)  to  Commander’s  making  decisions  on 
the  battlefield. 

3.  Management  Value  Added  (MVA)  research  should  continue  that 
will  evaluate  the  future  of  management  in  organizations  such  as  the 
Department  of  Defense  preventing  the  risk  of  poor  job  performance, 
new  product  delivery  time,  and  poor  acquisition  cost  estimates. 
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C.  RECOMMENDATIONS 

MDA  should  move  to  the  next  step,  meaning  if  you  want  a  continuous  ROI 
results,  you  have  to  monitor  the  process  overtime  and  continually  compute  the 
results  against  the  KVA+RO  framework  versus  sampling  only  one  time  as  we  did 
in  this  case  study.  The  continuous  sampling  will  enable  decision  makers  to  make 
a  more  informed  decision  with  all  options  considered  against  aggregate 
performance  feedback.  MDA  leadership  is  critical  in  helping  select  the  options 
for  improvement  of  the  MDA  process. 

MDA  Subject  Matter  Experts  (SME)  should  utilize  capable  software  such 
as  GaussSoft  that  would  provide  a  more  proficient  KVA  output  for  continuous 
monitoring  of  MDA  processes  in  the  near  future. 
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VII.  CONCLUSION 


The  proof  of  concept  study  demonstrated  the  research  approach  can  be 
used  to  value  changes  through  automation  to  the  MDA  process  of  Track 
generation  and  it  helps  top  level  management  make  portfolio  management 
decisions  for  allocating  resources  for  MDA  information  systems  or  reallocation 
based  on  the  ability  to  exercise  the  most  beneficial  option  for  the  organization. 
Track  generation  automation  helps  provide  the  following  improvements: 

•  faster  Track  generation  of  a  Maritime  Contact  of  Interest  (COI) 

•  provide  the  opportunity  to  improve  fleet  Maritime  Interdiction 
Operations  (MIO)  intelligence  gathering 

•  decrease  personnel  costs  by  reducing  the  number  of  MDA  watch 
standers 

•  improve  productivity  in  current  MDA  processes,  allowing  more  U.S. 
Navy  and  U.S.  Coast  Guard  ship  boardings 

This  new  Track  generation  automation  will  help  provide  increased  value  in 
Maritime  Domain  Awareness  (MDA).  The  technology  implementation  presents  a 
great  opportunity  for  the  Department  of  Homeland  Security,  U.S.  Coast  Guard, 
and  U.S.  Navy  to  maintain  their  fight  on  the  Global  War  on  Terrorism  when 
conducting  Maritime  Interdiction  Operations  (MIO)  during  peacetime  and  wartime 
as  the  threat  of  terrorism  becomes  more  unpredictable. 
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APPENDIX  A.  AS  IS 


Table  6.  KVA  AS  IS  Data 
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APPENDIX  B.  TO-BE 


Table  7.  KVA  Actual  To-Be  Data 
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APPENDIX  C.  RADICAL  TO-BE 


Table  8.  KVA  Radical  To-Be  Data 
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